One or more embodiments of the invention generally relate to data compression. More particularly, certain embodiments of the invention relates to mask based compression scheme.
The following background information may present examples of specific aspects of the prior art (e.g., without limitation, approaches, facts, or common wisdom) that, while expected to be helpful to further educate the reader as to additional aspects of the prior art, is not to be construed as limiting the present invention, or any embodiments thereof, to anything stated or implied therein or inferred thereupon.
A typical neural network may use input data and weight to classify an object. The weight and feature map typically may be large. In numerical analysis, a sparse matrix is a matrix in which most of the elements are zero. By contrast, if most of the elements are nonzero, then the matrix is considered dense. The number of zero-valued elements divided by the total number of elements is called the sparsity of the matrix. When storing and manipulating sparse matrices on a computer, it may be beneficial and often necessary to use specialized algorithms and data structures that take advantage of the sparse structure of the matrix. Operations using standard dense-matrix structures and algorithms are believed to be slow and inefficient when applied to large sparse matrices as processing and memory are wasted on the zeroes. Sparse data is by nature more easily compressed and thus require significantly less storage.
The following is an example of a specific aspect in the prior art that, while expected to be helpful to further educate the reader as to additional aspects of the prior art, is not to be construed as limiting the present invention, or any embodiments thereof, to anything stated or implied therein or inferred thereupon. The Compressed Sparse Row (CSR) or Compressed Row Storage (CRS) format represents a matrix by three (one-dimensional) arrays, that respectively contain nonzero values, the extents of rows, and column indices. The Compressed Sparse Column (CSC) is similar to CSR except that values are read first by column, a row index is stored for each value, and column pointers are stored. It is believed that they are may not be very efficient, needing an indirect addressing step for every single scalar operation in a matrix-vector product or preconditioner solve.
In view of the foregoing, it is clear that these traditional techniques are not perfect and leave room for more optimal approaches.
Unless otherwise indicated illustrations in the figures are not necessarily drawn to scale.